Integration of GCP with nOps

FAQs

Expand FAQs

1. What if I don’t see my billing data in BigQuery?

• Ensure that Billing Export is enabled for the correct project.
• Check if the dataset and table names match what was configured in nOps.
• Wait up to 48 hours for the first full dataset to appear.

2. Can I use customer-managed encryption keys?

No, Google Cloud only supports Google-managed encryption for billing data exports.

3. How often is billing data updated on nOps?

Billing data is refreshed daily, but there might be a 48-hour delay for full updates.

4. How long does it take for the Pricing and Detailed Resource tables to be available?

The tables under the Detailed Usage Cost and Pricing exports may take up to 48 hours to be created.

5. Do all exports need to be in the same location?

Yes. The Detailed Usage Cost and Pricing exports must be configured to use the same location (US or EU) and preferably the same project. Using different locations will cause integration issues with nOps.

6. Why do I need to grant permissions at three different levels (Organization, Billing Account, Project)?

Each level provides different types of access:

• Organization-level roles allow nOps to discover and analyze resources across all projects in your organization (e.g., Compute instances, GKE clusters, recommendations).
• Billing Account-level role is required to access billing metadata and currency information. This cannot be granted at the project level.
• Project-level roles provide access to the BigQuery dataset containing your cost export data.

Granting roles only at one level will result in incomplete data or integration failures.

Setting Up the Integration

To integrate GCP with nOps, follow these steps:

Prerequisites

Before proceeding, ensure you have completed the Prerequisites including:

• Checking if billing exports are already enabled
• Configuring GCP billing exports (if not already enabled)
• Understanding domain restricted sharing requirements (if applicable)

---

1. Link GCP Billing Data to nOps

Watch: Linking GCP Billing Data to nOps

Once billing exports are configured in GCP, connect them to nOps.

1. Log in to nOps and navigate to the GCP Integration page.
2. Enter the required information:
   • GCP Billing Account ID
   • BigQuery Dataset ID for Detailed Usage Cost
   • BigQuery Dataset ID for Pricing Export
   • BigQuery Dataset ID for Committed Use Discounts Export
3. Click Create Integration.

Finding Your IDs

Billing Account ID:

Console

Found on the Billing page (Format: 0115B9-C18400-A979DC)

Billing Account ID shown in the console

gcloud CLI

gcloud billing accounts list

BigQuery Dataset IDs:

1. Go to Billing > Billing Export. You may need to select your billing account first if prompted.
2. For each export (Detailed Usage Cost, Pricing, and Committed Use Discounts), click on the Dataset name link to open the dataset in BigQuery.

Billing Export page with dataset links

3. In BigQuery, click on the three vertical dots (⋮) next to the dataset name.
4. Select Copy ID to copy the full Dataset ID (e.g., my-billing-project.gcp_billing_exports).

Click the three dots menu and select Copy ID

Tip: If all exports share the same dataset, you only need to copy the ID once — use the same Dataset ID for all fields in nOps.

After completing this step, nOps will generate a service account email. Copy this email address — you will need it in the next section to grant permissions.

Copy the generated nOps service account email

---

2. Grant Service Account Permissions in GCP

Once the nOps service account email is generated, grant it the required permissions. Permissions must be granted at three different levels: Organization, Billing Account, and Project.

Important

The nOps service account requires permissions at multiple levels. Granting roles only at the project level will not provide sufficient access for full cost visibility and recommendations.

Domain Restricted Sharing

If your GCP organization has Domain Restricted Sharing enabled, you may need to add the nOps Customer ID to your allowed domains before granting permissions. See the Prerequisites page for detailed instructions.

---

A. Organization-Level Roles

Watch: Granting Organization IAM Roles

These roles provide visibility across your entire GCP organization for asset discovery, recommendations, and resource enumeration.

Required Roles:

• Cloud Asset Viewer (roles/cloudasset.viewer) – To enumerate assets across services for correlation.
• Browser (roles/browser) – To enumerate projects and folders.
• Recommender Viewer (roles/recommender.viewer) – To read cost recommendations (e.g., rightsizing, idle resources).
• Logs Viewer (roles/logging.viewer) – To read logs for resource analysis.
• Compute Viewer (roles/compute.viewer) – To read Compute Engine data (CUDs, instances, regions).
• Cloud SQL Viewer (roles/cloudsql.viewer) – To read Cloud SQL instances and configurations.
• Cloud Run Viewer (roles/run.viewer) – To read Cloud Run services and configurations.

Console

Step 1: Navigate to Organization IAM

1. Go to IAM & Admin → IAM in the Google Cloud Console.
2. At the top of the page, use the project/organization selector to switch to your Organization (not a specific project).

Select your Organization from the dropdown

Step 2: Grant Access to nOps Service Account

1. Click + Grant Access.
2. In the New principals field, enter the nOps service account email (obtained from the integration setup).
3. Add all the roles listed above.
4. Click Save.

Add the service account and assign all required roles

gcloud CLI

Set your configuration variables:

# Get your Organization ID
gcloud organizations list

# Replace with your values
NOPS_SERVICE_ACCOUNT="your-nops-sa@project.iam.gserviceaccount.com"
ORGANIZATION_ID="123456789012"  # Use the ID from the command above

Grant the required roles:

# Organization-level roles for fleet-wide visibility
ORG_ROLES=(
    "roles/cloudasset.viewer"
    "roles/browser"
    "roles/recommender.viewer"
    "roles/logging.viewer"
    "roles/compute.viewer"
    "roles/container.viewer"
    "roles/cloudsql.viewer"
    "roles/run.viewer"
)

echo "Granting Organization-level roles..."
for ROLE in "${ORG_ROLES[@]}"; do
    echo "  Granting $ROLE at org level..."
    gcloud organizations add-iam-policy-binding $ORGANIZATION_ID \
        --member="serviceAccount:$NOPS_SERVICE_ACCOUNT" \
        --role="$ROLE" \
        --quiet
done

---

B. Billing Account-Level Role

Watch: Granting Billing Account IAM Role

This role is required for currency and billing metadata validation. It must be granted directly on the Billing Account.

Required Role: Billing Account Viewer (roles/billing.viewer)

note

Permissions granted in the standard Project IAM page do not propagate to the Billing Account. You must grant this role via the Billing console or gcloud CLI.

Console

Step 1: Navigate to Billing Account Management

1. Go to Billing in the Google Cloud Console.
2. From the billing accounts list, click on the billing account name that you configured the exports for.
3. Click Account Management in the left-hand menu.

Navigate to Account Management in the left menu

Step 2: Open the Info Panel

1. On the Account Management page, click Show info panel in the top-right corner (if the panel is not already visible).
2. This will open a side panel titled with your billing account name.

Click Show info panel to reveal the permissions sidebar

Step 3: Grant Billing Account Viewer Role

1. In the info panel on the right, click + Add Principal.
2. In the New principals field, enter the nOps service account email.
3. Select the role: Billing Account Viewer (roles/billing.viewer).
4. Click Save.

Add the service account with Billing Account Viewer role

gcloud CLI

# Get your Billing Account ID
gcloud billing accounts list

# Replace with your values
NOPS_SERVICE_ACCOUNT="your-nops-sa@project.iam.gserviceaccount.com"
BILLING_ACCOUNT_ID="XXXXXX-XXXXXX-XXXXXX"  # Use the ID from the command above

Grant the Billing Account Viewer role:

gcloud billing accounts add-iam-policy-binding $BILLING_ACCOUNT_ID \
    --member="serviceAccount:$NOPS_SERVICE_ACCOUNT" \
    --role="roles/billing.viewer" \
    --quiet

---

C. Dataset-Level Roles (Cost Export Datasets)

Watch: Granting Dataset Permissions

These roles are required for reading BigQuery billing data and must be granted on each dataset that contains your cost export data:

• Detailed Usage Cost dataset
• Pricing dataset
• Committed Use Discounts dataset

Required Role: BigQuery Data Viewer (roles/bigquery.dataViewer)

Same Dataset?

If you configured all exports to use the same dataset, you only need to grant the permissions once on that shared dataset. Skip the other dataset steps if they share the same dataset as Detailed Usage Cost.

Step 1: Navigate to Billing Export

1. Go to Billing in the Google Cloud Console.
2. From the billing accounts list, click on the billing account name that you configured the exports for.
3. Select Billing Export from the left-hand menu.
4. On the Billing Export page, you'll see the Dataset name links for your configured exports.

Click on the dataset name to open it in BigQuery

Step 2: Open Dataset and Manage Permissions

1. Click on the Dataset name link for the first export (e.g., Detailed Usage Cost). This will open BigQuery with that dataset selected.
2. In BigQuery, click on the three vertical dots (⋮) next to the dataset name in the left panel.
3. Hover over Share and select Manage permissions.

Click the ⋮ menu → Share → Manage permissions

Step 3: Grant BigQuery Data Viewer Role

1. In the permissions panel, click Add Principal.

Click Add Principal

2. In the New principals field, enter the nOps service account email.
3. In the Select a role dropdown, choose BigQuery Data Viewer (roles/bigquery.dataViewer).

Select BigQuery Data Viewer role

4. Click Save.

Step 4: Repeat for Other Datasets (if different)

If your Pricing or Committed Use Discounts exports use different datasets than Detailed Usage Cost, go back to the Billing Export page and repeat Steps 2-3 for each additional export dataset.

info

If all exports share the same dataset, you can skip this step — permissions are already granted.

---

D. Project-Level Service Usage Role

The Service Usage Consumer role must be granted on the project that hosts your billing exports.

Required Role: Service Usage Consumer (roles/serviceusage.serviceUsageConsumer)

Console

1. Go to IAM & Admin → IAM in the Google Cloud Console.
2. Select the project that hosts your billing exports (where your BigQuery billing export dataset is located).
3. Click + Grant Access.

Grant Service Usage Consumer at project level

4. In the New principals field, enter the nOps service account email.
5. Select the role: Service Usage Consumer (roles/serviceusage.serviceUsageConsumer).

Select Service Usage Consumer role

6. Click Save.

gcloud CLI

# Replace with your values
NOPS_SERVICE_ACCOUNT="your-nops-sa@project.iam.gserviceaccount.com"
BILLING_EXPORT_PROJECT_ID="your-billing-export-project"  # The project hosting your billing exports

Grant the Service Usage Consumer role:

gcloud projects add-iam-policy-binding $BILLING_EXPORT_PROJECT_ID \
    --member="serviceAccount:$NOPS_SERVICE_ACCOUNT" \
    --role="roles/serviceusage.serviceUsageConsumer" \
    --quiet

---

E. Project-Level BigQuery Resource Viewer Role (Optional)

Watch: Granting BigQuery Resource Viewer Role

When is this needed?

This role is only required if your organization uses flat-rate or reservation-based BigQuery pricing. Most customers use on-demand pricing and can skip this section.

How to check: Go to BigQuery → Administration → Reservations in the GCP Console. If you see reservations or capacity commitments configured, you need this role.

The BigQuery Resource Viewer role allows nOps to read BigQuery reservation and slot information for capacity planning and cost optimization of flat-rate BigQuery workloads.

Role: BigQuery Resource Viewer (roles/bigquery.resourceViewer)

Console

1. Go to IAM & Admin → IAM in the Google Cloud Console.
2. Select the project that contains your BigQuery billing export dataset.
3. Click + Grant Access.

Grant BigQuery Resource Viewer at project level

4. In the New principals field, enter the nOps service account email.
5. Select the role: BigQuery Resource Viewer (roles/bigquery.resourceViewer).

Select BigQuery Resource Viewer role

6. Click Save.

gcloud CLI

# List your projects
gcloud projects list

# Replace with your values
NOPS_SERVICE_ACCOUNT="your-nops-sa@project.iam.gserviceaccount.com"
PROJECT_ID="your-billing-project"  # Use the project containing your billing export dataset

Grant the BigQuery Resource Viewer role:

gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member="serviceAccount:$NOPS_SERVICE_ACCOUNT" \
    --role="roles/bigquery.resourceViewer" \
    --quiet

---

3. Enable Required APIs

Enable the following APIs to allow nOps to collect cost and usage data. APIs only need to be enabled in the project that hosts your billing exports, not across all projects.

Simplified Setup

The Cloud Asset API can discover resources across your entire organization from a single project. This means you don't need to enable individual service APIs (like Cloud SQL or Cloud Run) in every project.

Required APIs:

API	Service ID	Purpose
Cloud Asset API	cloudasset.googleapis.com	Asset inventory and discovery across your organization
Cloud Billing API	cloudbilling.googleapis.com	Billing account access and cost data
Recommender API	recommender.googleapis.com	Cost optimization recommendations

Optional APIs:

API	Service ID	When Required
BigQuery Reservation API	bigqueryreservation.googleapis.com	Only if using flat-rate/reservation BigQuery pricing (for capacity commitments)

Console

Step 1: Open APIs & Services

1. Go to APIs & Services → Library in the Google Cloud Console.
2. Select the project that hosts your billing exports from the top dropdown menu.

Select the project that hosts your billing exports from the dropdown

Step 2: Enable Required APIs

Enable the following APIs in the project that hosts your billing exports:

1. Search for Cloud Asset API and click Enable.
2. Search for Cloud Billing API and click Enable.
3. Search for Recommender API and click Enable.

Search for each API and click Enable

Optional: BigQuery Reservation API

Only enable the BigQuery Reservation API if you use flat-rate or reservation-based BigQuery pricing. Most customers use on-demand pricing and can skip this.

gcloud CLI

Enable Required APIs

# Replace with your project ID (the project hosting your billing exports)
BILLING_EXPORT_PROJECT_ID="your-billing-export-project"

# Enable required APIs in the billing export project
gcloud services enable \
    cloudasset.googleapis.com \
    cloudbilling.googleapis.com \
    recommender.googleapis.com \
    --project=$BILLING_EXPORT_PROJECT_ID

Optional: Enable BigQuery Reservation API

# Only if you use flat-rate/reservation BigQuery pricing
gcloud services enable bigqueryreservation.googleapis.com \
    --project=$BILLING_EXPORT_PROJECT_ID

Terraform

For organizations with many projects, we recommend using the nOps GCP API Enablement Terraform Module. This module programmatically enables the required APIs in your billing export project and optionally across your organization.

module "enable_gcp_apis" {
  source = "github.com/nops-io/terraform-gcp-nops-integration"
  
  organization_id            = "123456789012"
  billing_export_project_id  = "my-billing-export-project"
}

See the full Terraform Setup Guide →

---

Why Use nOps for GCP Cost Management?

• Unified Cost Visibility – View all cloud spend, including AWS, Azure, and GCP, in a single platform.
• Automated Cost Analysis – Identify inefficiencies and optimize resource allocation.
• Custom Reporting – Create tailored reports for detailed GCP spend analysis.

By integrating GCP with Inform, you gain deeper visibility into cloud costs, empowering smarter budget decisions.